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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH ) 

ASSOCIATION, INC., AMERICAN ) 

PSYCHOLOGICAL ASSOCIATION, INC., ) 

and NATIONAL COUNCIL ON ) 

MEASUREMENT IN EDUCATION, INC., ) Civil Action No. 1 :14-cv-00857-TSC-DAR 

) 

Plaintiffs, ) DECLARATION OF LAURESS 

) L. WISE IN SUPPORT OF 

V. ) PLAINTIFFS’ MOTION FOR 

) SUMMARY JUDGMENT AND ENTRY 
PUBLIC.RESOURCE.ORG, INC., ) OF A PERMANENT INJUNCTION 

) 

Defendant. ) 

) 

I, LAURESS L. WISE, declare: 

1 . I am the Immediate Past President of the National Council on Measurement in 
Education, Inc. (“NCME”). I have been a member of this organization for approximately 30 
years. I previously was the President of NCME from April 2014 through April 2015, and Vice 
President of this organization from April 2013 through April 2014. I submit this Declaration in 
support of the motion of the American Educational Research Association, Inc. (“AERA”), the 
American Psychological Association, Inc. (“APA”), and the NCME (collectively, “Plaintiffs” or 
“Sponsoring Organizations”) for summary judgment and the entry of a permanent injunction. 

2. I also am a principal scientist with the Human Resources Research Organization 
(“HumRRO”), spending full time on research and evaluation projects relating to educational 
measurement. I previously served as HumRRO CEO for 13 years, combining management and 
research activities and, before that, directed research and development for the Armed Services 
Vocational Aptitude Battery for the Department of Defense. Before that I spent 16 years as a 
researcher for the American Institutes for Research, rising to the position of Director of 
Research. I am also a member of both AERA and APA. 
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3. NCME is a District of Columbia not-for-profit corporation. 

4. NCME is a professional organization for individuals involved in assessment, 
evaluation, testing, and other aspects of educational measurement. NCME’s members are 
involved in the construction and use of standardized tests; new forms of assessment, including 
performance-based assessment; program design; and program evaluation. 

5. In 1955, AERA and NCME prepared and published a companion document to 
APA’s “Technical Recommendations for Psychological Tests and Diagnostic Techniques” 
(published in 1954), entitled “Technical Recommendations for Achievement Tests.” 

6. Subsequently, a joint committee of the three organizations modified, revised and 
consolidated the two documents into the first Joint Standards. Beginning with the 1966 revision, 
the Sponsoring Organizations collaborated in developing the “Joint Standards” (or simply, the 
“Standards”). Each subsequent revision of the Standards has been careful to note that it is a 
revision and update of that document. 

7. Beginning in the mid-1950s, the Sponsoring Organizations formed and 
periodically reconstituted a committee of experts in psychological and educational assessment, 
charged with the initial development of the Technical Recommendations and then each 
subsequent revision of the (renamed) Standards. These committees were formed by the three 
organizations’ Presidents (or their designees), who would meet and jointly agree on the 
membership. Often a chair or co-chairs of these committees were selected by joint agreement. 
Beginning with the 1966 version of the Standards, this committee became referred to as the 
“Joint Committee.” For example, I was the co-chair of the Joint Committee for the 2014 edition 
of the Standards. 

8. Financial and operational oversight for the Standards’ revisions, promotion. 
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distribution, and for the sale of the 1999 and 2014 Standards has been undertaken by a 
periodically reconstituted Management Committee, comprised of designees of the three 
Sponsoring Organizations. 

9. All members of the Joint Committee(s) and the Management Committee(s) are 
unpaid volunteers. The expenses associated with the ongoing development and publication of 
the Standards include travel and lodging expenses (for the Joint Committee and Management 
Committee members), support staff time, printing and shipment of bound volumes, and 
advertising costs. 

10. Many different fields of endeavor rely on assessments. The Sponsoring 
Organizations have ensured that the range of these fields of endeavor is represented in the Joint 
Committee’s membership - e.g., admissions, achievement, clinical counseling, educational, 
licensing-credentialing, employment, policy, and program evaluation. Similarly, the Joint 
Committee’s members represent expertise across major functional assessment areas - e.g., 
validity, equating, reliability, test development, scoring, reporting, interpretation, large scale 
interpolation and cognitive behavioral therapy. 

11. From the time of their initial creation to the present, the preparation and periodic 
revisions to the Standards entail intensive labor and considerable cross-disciplinary expertise. 
Each time the Standards are revised, the Sponsoring Organizations select and arrange for 
meetings of the leading authorities in psychological and educational assessments (known as the 
Joint Committee). During these meetings, certain Standards are combined, pared down, and/or 
augmented, others are deleted altogether, and some are created as whole new individual 
Standards. The 1999 version of the Standards is nearly 200 pages, and took more than five years 
to complete - resulting from work put in by the Joint Committee to generate a set of best 
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practices on educational and psychological testing that are respected and relied upon by leaders 
in their fields. 

12. The Standards originally were created as principles and guidelines - a set of best 
practices to improve professional practice in testing and assessment across multiple settings, 
including education and various areas of psychology. The Standards can and should be used as a 
recommended course of action in the sound and ethical development and use of tests, and also to 
evaluate the quality of tests and testing practices. Additionally, an essential component of 
responsible professional practice is maintaining technical competence. Many professional 
associations also have developed standards and principles of technical practice in assessment. 
The Sponsoring Organizations’ Standards have been and still are used for this purpose. 

13. The Standards, however, are not simply intended for members of the Sponsoring 
Organizations, AERA, APA, and NOME. The intended audience of the Standards is broad and 
cuts across audiences with varying backgrounds and different training. For example, the 
Standards also are intended to guide test developers, sponsors, publishers, and users by providing 
criteria for the evaluation of tests, testing practices, and the effects of test use. Test user 
standards refer to those standards that help test users decide how to choose certain tests, interpret 
scores, or make decisions based on tests results. Test users include clinical or industrial 
psychologists, research directors, school psychologists, counselors, employment supervisors, 
teachers, and various administrators who select or interpret tests for their organizations. There is 
no mechanism, however, to enforce compliance with the Standards on the part of the test 
developer or test user. The Standards, moreover, do not attempt to provide psychometric 
answers to policy or legal questions. 

14. The Standards promote the development of high quality tests and the sound use of 
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results from such tests. Without such high quality standards, tests might produce scores that are 
not defensible or accurate, not an adequate reflection of the characteristic they were intended to 
measure, and not fair to the person tested. Consequently, decisions about individuals made with 
such test scores would be no better, or even worse, than those made with no test score 
information at all. Thus, the Standards help to ensure that measures of student achievement are 
relevant, that admissions decisions are fair, that employment hiring and professional 
credentialing result in qualified individuals being selected, and patients with psychological needs 
are diagnosed properly and treated accordingly. Quality tests protect the public from harmful 
decision making and provide opportunities for education and employment that are fair to all who 
seek them. 

15. The Standards apply broadly to a wide range of standardized instruments and 
procedures that sample an individual’s behavior, including tests, assessments, inventories, scales, 
and other testing vehicles. The Standards apply equally to standardized multiple-choice tests, 
performance assessments (including tests comprised of only open-ended essays), and hands-on 
assessments or simulations. The main exceptions are that the Standards do not apply to 
unstandardized questioimaires (e.g., unstructured behavioral checklists or observational forms), 
teacher-made tests, and subjective decision processes (e.g., a teacher’s evaluation of students’ 
classroom participation over the course of a semester). 

16. The Standards have been used to develop testing guidelines for such activities as 
college admissions, persormel selection, test translations, test user qualifications, and computer- 
based testing. The Standards also have been widely cited to address technical, professional, and 
operational norms for all forms of assessments that are professionally developed and used in a 
variety of settings. The Standards additionally provide a valuable public service to state and 
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federal governments as they voluntarily choose to use them. For instance, each testing company, 
when submitting proposals for testing administration, instead of relying on a patchwork of local, 
or even individual and proprietary, testing design and implementation criteria, may rely instead 
on the Sponsoring Organizations’ Standards to afford the best guidance for testing and 
assessment practices. 

1 7. The Standards were not created or updated to serve as a legally binding document, 
in response to an expressed governmental or regulatory need, nor in response to any legislative 
action or judicial decision. However, the Standards have been cited in judicial decisions related 
to the proper use and evidence for assessment, as well as by state and federal legislators. These 
citations in judicial decisions and during legislative deliberations occurred without any lobbying 
by the Plaintiffs. 

18. NCME has never communicated with Congress for the purpose of encouraging 
the enactment of the Standards into law. 

19. Additionally, NCME has never solicited any government agency to incorporate 
the Standards into the Code of Federal Regulations or other rules of Federal or State agencies. 

20. In the policymaking arena, NCME believes the Standards should be treated as 
guidelines informing the enactment of legislation and regulations consistent with best practices 
in the development and use of tests - to insure that they are valid, reliable and fair. 

21. The Sponsoring Organizations promote and sell copies of the Standards via 
referrals to the AERA website, at annual meetings, in public offerings to students, and to 
educational institution faculty. Advertisements promoting the Standards have appeared in 
meeting brochures, in scholarly journals, and in the hallways at professional meetings. 
Accompanying this Declaration as Exhibit KKK is a true copy of advertisements for the 1999 
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Standards published in NCME’s Journal of Educational Management. These advertisements 
were produced at Bates Nos. AERA_APA_NCME_003 1444-003 1451. 

22. Distribution of the Standards is closely monitored by the Sponsoring 
Organizations. AERA, the designated publisher of the Standards, sometimes does provide 
promotional complementary print copies to students or professors. Except for these few 
complementary print copies, however, the Standards are not given away for free; and certainly 
they are not made available to the public by any of the three organizations for anyone to copy 
free of charge. 

23. To date, NCME has never posted, or authorized the posting of, a digitized copy of 
the 1 999 Standards on any publicly accessible website. 

24. Without receiving at least some moderate income from the sales of the Standards 
to offset their production costs and to allow for further revisions, it is very likely that the 
Sponsoring Organizations would no longer undertake to periodically update them, and it is 
unknown who else would. 

25. In late 2013 and early 2014, the Sponsoring Organizations became aware that the 
1999 Standards had been posted on the Internet without their authorization, and that students 
were obtaining free copies from the posting source. Upon further investigation, the Sponsoring 
Organizations discovered that Public Resource was the source of the online posting. 

26. Public Resource posted Plaintiffs’ 1 999 Standards to its website and the Internet 
Archive website without the permission or authorization of any of the Sponsoring Organizations. 

27. Plaintiffs have been made aware that at least some of those users who obtained 
the 1999 Standards for free from Public Resource did so to avoid paying the modest sale price 
for authorized print copies. 
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28. Accompanying this Declaration as Exhibit LLL is a true copy of an e-mail dated 
Mareh 5, 2014 from Gregory J. Cizek to me regarding a student not purchasing the 1999 
Standards because “they [were] available for free online” at 
https://law.resource.org/pub/us/cfr/ibr/001/aera.standards.1999.pdf.” This e-mail exchange was 
marked as Exhibit 1252 during my deposition. 

I DECLARE, under the penalty of pequry, that the foregoing is true and correct. 


Dated: Deeember ^ . 2015 




Lauress L. Wise 
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EXHIBIT 1 

LAURESS L. WISE 

Curriculum Vitae 

OVERVIEW 


Dr. Lauress Wise has over 35 years’ experience in educational research and continues 
extensive work on educational policy and assessment issues. Dr. Wise currently advises 
several states and the PARCC assessment consortium on technical issues in test development 
and use. He serves on the Board of the National Council of Measurement in Education as the 
immediate past-president. He is also serving on a National Research Council Committee that is 
evaluating the NAEP achievement levels. He recently co-chaired the panel that revise the 
AERA/APA/NCME Standards for Educational and PsychologicalTestlng and previously chaired 
the National Academy of Sciences Board on Testing and Assessment. Recent research and 
development efforts include a 15-year independent evaluation of the California High School Exit 
Exam and quality assurance work for the National Assessment of Educational Progress 
(NAEP). Dr. Wise previously served on several National Research Council committees, 
chairing the Committees on Scientific Research in Education and the Evaluation of the National 
Voluntary Tests. 


EDUCATION 


Ph.D, Mathematical Psychology 1975 University of California, Berkeley 

B.S., Mathematics, Psychology (with Distinction) 1967 Stanford University 


AREAS OF EXPERTISE 


Test Development and Validation 
Program and Policy Evaluation 
Test Use Policy 


Project Management 

Statistical and Psychometric Issues 

Computer-Based Testing 


PROFESSIONAL EXPERIENCE 


Human Resources Research Organization 1994 - 2015 

Principal Scientist 

• Served as HumRRO’s president from 1994 to 2007. Remained active in research on testing 
and test use policy. Directed two major HumRRO educational testing projects, one to 
provide quality assurance for the National Assessment of Educational Progress (NAEP) and 
the other an independent evaluation of California’s High School Exit Exam (CAHSEE). He 
continues to serve as a senior psychometric advisor for a graduate school admissions 
testing program. 

• Served as co-chair of the committee that revised the 1999 AERA/APA/NCME Standards for 
Educational and Psychological Testing, and has previously served as Chair of the National 
Academy of Science (NAS) Board on Testing and Assessment and chaired the NAS 
Committee on Research in Education. 

• Currently serves on technical advisory committees for the Hawaii, Wyoming, Utah, 
Tennessee, and Virginia departments of education, and the Partnership for Assessing 
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Readiness for College and Career (PARCC) advisory committees. Also serves on the 
Rhode Island Technical Advisory Committee for Teacher Evaluation. 

• Served as co-Principal Investigator on the first year of the Congressionally-mandated 
evaluation of President Clinton’s Voluntary National Tests and chaired the NAS committee 
that performed the second year of that evaluation, and on the NAS committee to evaluate 
the NAEP and on the National Academy of Education's Panel for the Evaluation of the 
NAEP Trial State NAEP. 

• Other work includes vertical alignment of state content standards, modeling the effects of 
motivation on examinee performance on low-stakes assessments, the impact of changes in 
exclusions on NAEP results for Kentucky, and scaling constructed response and multiple 
choice items on the Florida assessment. Dr. Wise also worked on the development and 
validation of a computer-administered assessment now used for selection of air traffic 
controllers and developed a computer-based system for assessing work values as part of a 
Department of Labor effort to develop improved career guidance tools. 

Defense Manpower Data Center 1990 - 1994 

Chief, Personnel Testing Division 

• Spokesperson for the Department of Defense on matters relating to the development and 
use of cognitive tests. Dr. Wise’s unit was responsible for all research and development for 
the Armed Services Vocational Aptitude Battery (ASVAB). 

• Work included evaluation and implementation of a computerized, adaptive version of the 
ASVAB, automated item and form development procedures, new career exploration 
procedures for use with the high school testing program, the development and testing of a 
new career interest inventory, and extensive validity research. 

American Institutes for Research (AIR) 1974 - 1990 

Associate Research Scientist to Director of Research 

• Directed a variety of studies and projects, including the Review and Analysis of the General 
Aptitude Test Battery (GATB) Project for the U.S. Employment Service within the 
Department of Labor, the Army Synthetic Validation Project, the Army's Computerized 
Adaptive Screening Test (CAST) Revision Project, and validation studies for the Software 
Engineering Institute at Carnegie Mellon University. 

• Served as director of analysis for the Army's massive Project A, analyzing new selection 
tests, developing models of performance in a variety of jobs, and assessing the validity of 
each new test for predicting different facets of performance in different jobs. 

• Served for twelve years as the chief psychometrician for the Medical College Admissions Test, 
developing procedures for the screening and calibration of new items and for the construction 
and equating of new forms. Also consulted with the Department of Education on issues related 
to testing and data analysis as part of the Statistical Analysis Group in Education. 

• From 1978 to 1982, served as Director of Project TALENT, a nationally representative 
longitudinal study of nearly 400,000 members of the high school classes of 1960 through 
1963. Oversaw the collection of the final wave of follow-up data and conducted targeted 
research on issues such as gender differences in mathematics achievement, school 
differences in student achievement, the development of careers in science and medicine, 
and the consequences of adolescent childbearing. 
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University of California 

Programmer and Instructor 

• While a graduate student at the University of California, Dr. Wise served as the computer 
consultant for the Psychology Department, helping both faculty and students in the design and 
execution of data analyses. He also taught an undergraduate course in Psychological Statistics. 

California Department of Public Health 1968 - 1972 

Computer Programmer and Data Processing Systems Analyst 

• Created data systems to support licensing functions and vital statistics systems at the 
Department of Public Health. Was in-house project manager for a new management 
information system that involved defining "outputs" for each bureau and department and 
relating these outputs to costs. 


PROFESSIONAL AFFILIATIONS AND SERVICE 


• American Educational Research Association (AERA) 

• American Psychological Association (APA) 

Divisions 5, 14, 19 

• Member, National Council on Measurement in Education (NCME) 

• Psychometric Society 

SELECTED BIBLIOGRAPHY 


Publications 

Wise, L. L. (in press). How we got to where we are: Evolving policy demands for the next 
generation assessments. In Next Generation Assessments. R. Lissitz, Ed. (To be published in 
2016. 

Wise, L. L. & Plake, B. S. (2015). "Test design and development following the Standards for 
Educational and Psychological Testing". In Lane, S., Haladyna, T., and Raymond, M. (Eds.). 
Handbook of Test Development. New York, NY: Routledge. 

Plake, B. S., & Wise, L. L. (2014). Revision of the AERA, APA, NCME Standards for 
Educational and Psychological Testing: What is their role and importance for NCME. 
Educational Measurement: Issues and Practice, 33(4), 4-12. 

Wise, L. L. (2010). Accessible Reading Assessments for Students with Disabilities: Summary 
and Conclusions. Applied Measurement in Education, 23(2), 209-214. 

Wise, L. L. (2006). Encouraging and Supporting Compliance with Standards for Educational 
Tests. Educational Measurement: Issues and Practice 25(3), 51-53. 

National Research Council. (2005). Advancing Scientific Research in Education. Committee on 
Research in Education. Lisa Towne, Lauress L. Wise, and Tina M Winders, Editors. Center for 
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Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The 
National Academy Press. 

National Research Council. (2004). Strengthening Peer Review in Federai Agencies That 
Support Education Research. Committee on Research in Education. L. Towne, J.M. Fletcher, 
and L.L. Wise, Eds. Center for Education, Division of Behavioral and Social Sciences and 
Education. Washington, DC: The National Academies Press. 

Wise, L.L. (2004). The National Assessment of Educational Progress - what it tells educators. In 
J.E. Wall & G.R. Walz (Eds.). Measuring up: Assessment issues for teachers, counseiors, and 
administrators (pp. 729-741). Greensboro, NC: CAPS Press. 

Wise, L.L. & Hoffman, R.G. (2002). How wiii assessment data to be used to document the 
impact of educationai reform. In R. W. Lissitz and W. D. Schafer (Eds.) Assessment in 
Educationai Reform: Both means and ends. Boston, MA: Allyn & Bacon. 

Wise, L.L., Noeth, R.J., & Koenig, J.A. (Eds.) (1999). Evaiuation of the voiuntary nationai tests, 
year 2 interim report. National Research Council. Washington, DC: National Academy Press. 

Wise, L.L., Hauser, R.M., Mitchell, K.J., & Feuer, M.J. (1999). Evaiuation of the voiuntary 
nationai tests: Phase i. National Research Council, Commission on Behavioral and Social 
Sciences and Education, Board on Testing and Assessment. Washington, DC: National 
Academy Press. 

Wise, L.L., Curran, L.T. Curran, & McBride, J.R. (1997). CAT-ASVAB cost and benefit anaiyses. 
In W.A. Sands, B.K. Waters, & J.R. McBride (Eds.), Computerized Adaptive Testing: From 
Inquiry to Operation. Washington, DC: American Psychological Association. 

Wolfe, J.H., Alderton, D.L., Larson, G.E., Bloxom, B., & Wise, L.L. (1997). Expanding the 
content of CAT-ASVAB: New tests and their vaiidity. In W.A. Sands, B.K. Waters, & J.R. 
McBride (Eds.), Computerized Adaptive Testing: From Inquiry to Operation. Washington, DC: 
American Psychological Association. 

Wall, J.E., Wise, L.L., & Baker, H.E. (1996). Deveiopment of the interest-finder: A new RiASEC- 
based interest inventory. Measurement and Evaluation in Counseling and Development, 29, 
134-152. 

Wise, L.L. (1994). Goais of the seiection and ciassification decision. In M.G. Rumsey, C.B. 
Walker, & J.H. Harris (Eds.). Personnel selection and classification. Hillsdale, NJ: Lawrence 
Erlbaum Associates. 

Wise, L.L. (1994). Setting performance goais for the DOD Linkage Modei. In B.F. Green & A.S. 
Mavor (Eds.) Modeling cost and performance for military enlistment. Washington, DC: National 
Academy of Sciences Press. 

Rudner, L.M., Wise, L.L., & Stonehill, R.M. (1991). The ERIC Clearinghouse on Tests, 
Measurement, and Evaluation (ERIC/TME) — A growing resource. Applied Measurement in 
Education, 4, 1-10. 
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Wise, L.L. (1 991 ). The validity of test scores for selecting and classifying enlisted recruits. In 
B.R. Gifford & L.C. Wing (Eds.), Test policy in Defense: Lessons from the military for education, 
training, and employment. Boston, MA: Kluwer Academic Press. 

Campbell, J.P., McHenry, J.J., & Wise, L.L. (1990). Modeling Job performance in a population of 
jobs. Personnel Psychology, 43, 313-334. 

Young, W.Y., Houston, J.S., Harris, J.H., Hoffman, R.G., & Wise, L.L. (1990). Large-scale 
predictor validation in Project A: Data collection procedures and data base preparation. 
Personnel Psychology, 43, 301-312. 

Wise, L.L., McHenry, J.J., & Campbell, J.P. (1990). Identifying optimal predictor composites and 
testing for generalizability across jobs and performance factors. Personnel Psychology, 43, 355- 
366. 

Wise, L.L., Campbell, J.P., & Arabian, J.M. (1988). The Army Synthetic Validation Project. In 

B. F. Green, H. Wing, & A.K. Wigdor (Eds.) Linking military enlistment standards to job 
performance. Washington, DC: National Academy Press. 

Wise, L.L. (1985). Project TALENT: Mathematics course participation in the 1960's and its 
career consequences. In S.F. Chipman, L.R. Brush, & D.M. Wilson (Eds.), Women and 
mathematics: Balancing the equation. Hillsdale, NJ: Lawrence Erlbaum Associates. 

Abeles, R.P., Steel, L.M., & Wise, L.L. (1980). Patterns and implications of life-course 
organization: Project TALENT studies. In P.B. Baltes & O.G. Brim, Jr. (Eds.), Lifespan 
development and behavior (Vol. III). New York: Academic Press. 

Wise, L.L., & Steel, L.M. (1980). Educational attainment of the high school classes of 1960 
through 1963: Findings from Project TALENT. In A.C. Kerckhoff (Ed.), Longitudinal 
perspectives on educational attainment. Greenwich, CT: JAI Press. 

Wise, L.L. (1 979). Project TALENT : Studying the development of our human resource. In J.E. 
Milholland (Ed.), New directions for testing and measurement: Insights from large-scale 
surveys. San Francisco: Jossey-Bass Inc. 

Thacker, A. A., Dickinson, E. R., Bynum, B. H., Wen, Y., Smith, E. A., Sinclair, A. L., Deatz, R. 

C. , & Wise, L. L. (2015). Findings from the quality of items/tasks/stimuli investigations: PARCC 
field tests (2015 No. 009). Alexandria, VA: Human Resources Research Organization. 

Becker, D.E., Wise, L. L., Hardoin, M. M. & Watters, C. (2014). Independent evaluation of the 
California High School Exit Examination: 2014 Biennial report (2014 No. 001 ). Alexandria, VA: 
Human Resources Research Organization. 

Thacker, A. A., Dickinson, E. R., Wise, L. L., & Becker, D. E. (2014). PARCC studies to examine 
comparability of scores across states, assessment forms, scoring methods and other relevant 
variables memorandum (2014 No. 005). Alexandria, VA: Human Resources Research Organization. 

Thacker, A. A., Sinclair, A. L., Wise, L. L., & Becker, D. E., (2014). PARCC validity studies 
including predictive and longitudinal studies memorandum (2014 No. 020). Alexandria, VA: 
Human Resources Research Organization. 
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assurance for NAEP 2013 reading results (2013 No. 049). Alexandria, VA: Human Resources 
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control procedures and documentation: Fiscal Year 2011 (FR-1 1-17). Alexandria, VA: Human 
Resources Research Organization. 

Trippe, D. M., Waugh, G. W., Hoffman, R. G., McCloy, R. A., & Wise, L. L. (2010). Computer- 
based testing (CBT) feasibility study for the CFP^ certification examination (FR-1 0-78). 
Alexandria, VA: Human Resources Research Organization. 

Becker, D.E., Wise, L.L., Hardoin, M.M., & Watters, C. (Eds.) (2012). Independent evaluation of 
the California high school exit examination: 2011 evaluation report (FR-1 2-54). Alexandria, VA: 
Human Resources Research Organization. 

Thacker, A.A., Dickinson, E.R., Sinclair, A.L., & Wise, L.L. (2012). Independent review of test 
item quality for New York State Department of Education (NYSED) (FR-1 2-42). Alexandria, VA: 
Human Resources Research Organization. 

Wise, L.L., Thacker, A.A., Hoffman, R.G. (2012). Independent review of 2012 equating process 
(FR-1 2-37). Alexandria, VA: Human Resources Research Organization. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 15 of 26 


Becker, D.E., Wise, L.L., Hardoin, M.M., & Waters, C. (Eds.) (2012). \ndependent evaluation of 
the California High School Exit Examination: 2012 biennial report (FR-1 1-82). Alexandria, VA: 
Human Resources Research Organization. 

Becker, D.E., Wise, L.L., Hardoin, M.M., & Waters, C. (Eds.) (2011). \ndependent evaluation of 
the California High School Exit Examination: 2011 evaluation report (FR-1 1-51 ). Alexandria, VA: 
Human Resources Research Organization. 

Becker, D.E., Wise, L.L., & Sellman, W. S. (2011). Comprehensive review of NAEP quality 
control procedures and documentation: Fiscal Year 2011 (FR-1 1-17). Alexandria, VA: Human 
Resources Research Organization. 

Trippe, D.M., Waugh, G.W., Hoffman, R.G., McCloy, R.A., & Wise, L.L. (2010). Computer-based 
testing (CBT) feasibility study for the CFP® certification examination (FR-1 0-78). Alexandria, VA: 
Human Resources Research Organization. 

Becker, D.E., Wise, L.L., & Waters, C. (Eds.) (2010). \ndependent evaluation of the CAHSEE 
2010 evaluation report (FR-1 0-56). Alexandria, VA: Human Resources Research Organization. 

Hoffman, R.G., Wise, L.L., Diaz, T.E., Gribben, M.A., & Fry, S.C. (2009). Verification of NAEP 
grade 4 mathematics 2007 to 2009 trend results: Report of HumRRO activities (FR-09-66). 
Alexandria, VA: Human Resources Research Organization. 

Becker, D.E., Wise, L.L., & Watters, C. (Eds.) (2009). Independent evaluation of the California 
High Schocol Exit Examination (CAHSEE): 2009 evaluation report (FR-09-65). Alexandria, VA: 
Human Resources Research Organization. 

Gribben, M., Diaz, T., Wise, L.L. (2009). TUDA charter school study draft report: Base year 
special study {FR-09-13). Alexandria, VA: Human Resources Research Organization. 

Becker, D.E., Wise, L.L., & Watters, C. (Eds.) (2008). Independent evaluation of the California 
High School Exit Examination (CAHSEE): 2008 evaluation report (FR 08-100). Alexandria, VA: 
Human Resources Research Organization 

Gribben, M.A., Wise, L.L., & Becker, D.E. (2008). Review of web-based technical 
documentation process FY07 NAEP-QA special study report (TR-08-17). Alexandria, VA: 

Human Resources Research Organization. 

Wise, L.L., Becker, D.E., & Campbell, H. (2008). NAEP-QA review of procedures for mapping 
state performance standards onto the NAEP scale (TR-08-05). Alexandria, VA: Human 
Resources Research Organization. 

Wise, L.L., & Hoffman, R.G. (2007). HumRRO investigation of 2007 NAEP reading gains for 
American Indian/Alaskan native students (FR-07-85). Alexandria, VA: Human Resources 
Research Organization. 

Campbell, H.L., Wise, L.L., Becker, D.E., Campbell, C.H. (2007). NAEP-QA FY07 specialty 
study: Effects of expert scored papers (ESP) on scoring in the National Assessment of 
Educational Progress (FR-07-65). Alexandria, VA: Human Resources Research Organization. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 16 of 26 


Wise, L.L. (2007). Review of 2005 and 2006 Terra Nova results for the school district of 
Philadelphia (FR-07-18). Alexandria, VA: Human Resources Research Organization. 

Wise, L. L., Taylor, L. R., Becker, D. E., Gladden, F. B., Handy, K., Thacker, A. A., Schultz, S., 
Willison, S., & Dean, J. (2007). Development of performance level descriptors for the California 
Standards Tests (CSTs) and the High School Exit Examination (CAHSEE). (TR-07-01 ). 
Alexandria, VA: Human Resources Research Organization. 

Taylor, L. R., Wise, L. L., Thacker, A. A., Moody, R. L., Koger, L. E., Dickinson, E. R., & Trippe, 
D. M. (2007). Independent evaluation of the California Standards Tests (CSTs) and the 
California Alternate Performance Assessment (CAPA). (TR-07-04). Alexandria, VA: Human 
Resources Research Organization. 

Wise, L.L., Becker, D.E., Butler, F.L., Schantz, L., Bao, H., Sun, S., & Campbell, H.L. (2006). 
Independent evaluation of the California High School Exit Examination (CAHSEE): 2006 
evaluation report (FR-06-91). Alexandria, VA: Human Resources Research Organization. 

Stawarski, C., Teasdel, T., Wise, L.L., Schultz, S., Maloutas, M. (2006). National Assessment of 
Educational Progress (NAEP) State Coordinator Program Evaluation (IR-06-86). Alexandria, 

VA: Human Resources Research Organization. 

Becker, S., Thacker, A., Campbell, H., & Wise, L.L. (2006). Answering the billion dollar question: 
Are students learning more since the implementation of No Child Left Behind? Phase 1 report- 
draft (DFR 06-73). Alexandria, VA: Human Resources Research Organization. 

Diaz, T., Le, H., & Wise, LL (2006). NAEP-QA FY05 special study: 12^^ grade math trend 
estimates summary draft {FR-06-43). Alexandra, VA: Human Resources Research 
Organization. 

Sellman, W.S., Becker, D.E., Wise, L.L., & Hoffman, R.G. (2006). Notes from the meeting of the 
NAEP-Quality Assurance Consultant Panel, January 25-26, 2006 (SR-06-09). Alexandria, VA: 
Human Resources Research Organization. 

Wise, L.L., Le, H., Hoffman, R.G., & Becker, D.E. (2006). Testing NAEP full population 
estimates for sensitivity to violation of assumptions: Phase II (TR-06-08). Alexandria, VA: 

Human Resources Research Organization. 

Wise, L.L., Becker, D.E., Harris, C.D., Taylor, L.R., Johnstone, C.J., Miller, N.A., Thompson, 

S.J., Sun, S., Shen, X., Wang, X., Koger, L.E., & Moody, R. (2006). Independent evaluation of 
the California High School Exit Examination (CAHSEE): Third biennial report (FR-06-02). 
Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., Taylor, L.R., Wang, X., Becker, D.E., & Thacker, A. A. (2006). Review of the 
appropriateness of the California high school exit exam content standards for high school 
accountability {^R-06-0^). Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., & Becker, D.E. (2005). Independent Checks of 2005 NAEP State and TUDA Results: 
Reading and Math in Grades 4 and 8 (FR-05-67). Alexandria, VA: Human Resources Research 
Organization. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 17 of 26 


Becker, D.E., Diaz, T.E., Le, H., Shen, X., Hoffman, G.R., & Wise, L.W. (2005). Independent 
Checks of 2005 NAEP State and TUDA Results: Draft Report (FR-05-61). Alexandria, VA: 
Human Resources Research Organization. 

Wise, L.L., Becker, D.E., Harris, C.D., Taylor, L.R., Johnstone, C.J., Miller, N.A., Thompson, 

S.J., Sun, S., Shen, X., Butler, F.L., Wang, X., Koger, L.E., Moody, R., Deatz, R., Koger, M., 
Dickinson, E., Gensberg, S., Hilton, R.A., Kelley, N.L., & Stevens, C. (2005). Independent 
Evaluation of the California High School Exit Examination (CAHSEE) 2005 Evaluation Report 
(FR-05-43). Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., Zhang, L., Winter, P., Taylor, L., & Becker, D.E. (2005). Vertical Alignment of Grade- 
Level Expectations for Student Achievement: Report of a Pilot Study (TR-05-28). Alexandria, 

VA: Human Resources Research Organization. 

Wise, L.L., Becker, D.E., Harris, C.D., Sun, S., Wang, X., & Brown, D.G. (2004). Independent 
Evaluation of the California High School Exit Examination (CAHSEE): Year 5 Evaluation Report 
(FR-04-53). Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., Hoffman, R.G., & Becker, D.E. (2004). Testing NAEP full population estimates for 
sensitivity to violation of assumptions. Final Report (Draft). (TR-04-27). Alexandria, VA: Human 
Resources Research Organization. 

Wise, L.L., Harris, C.D., Koger, L.E., Bacci, E.D., Ford, J.P., Brown, D.G., Becker, D.E., Sun, S., 
Koger, M.E., Deatz, R.C., & Coumbe, K.L. (2004). Independent evaluation of the California High 
School Exit Examination (CAHSEE): Second biennial report (FR-04-01 ). Alexandria, VA: Human 
Resources Research Organization. 

Wise, L.L., Becker, D.E., & Gribben, M. (2003). Notes from the meeting of the National 
Assessment Governing Board, November 13-15, 2003 (SR-03-101). Alexandria, VA: Human 
Resources Research Organization. 

Wise, L. (2003). Notes from the meeting of the NAEP validity studies panel - September 12, 
2003 (SR-03-71). Alexandria, VA: Human Resources Research Organization. 

Hoffman, R.G., & Wise, L.L. (2003). NAEP quality assurance checks for the 2003 reading 
assessment results for grade 4 (FR-03-66). Alexandria, VA: Human Resources Research 
Organization. 

Wise, L.L., Harris, C.D., Brown, D.G., Becker, D.E., Sun, S., & Coumbe, K.L. (2003). California 
High School Exit Examination (CAHSEE): Year 4 evaluation report (FR-03-64r). Alexandria, VA: 
Human Resources Research Organization. 

Wise, L.L. (2003). Testing NAEP full population estimates for sensitivity to violation of 
assumptions - draft design and management plan (FR-03-61). Alexandria, VA: Human 
Resources Research Organization. 

Hoffman, R.G., & Wise, L.L. (2003). NAEP quality assurance special studies lll-B and lll-C: 

2003 data replication - final design and management plan (FR-03-59). Alexandria, VA: Human 
Resources Research Organization. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 18 of 26 


Hoffman, R.G., Wise, L.L., & Becker, D.E. (2003). NAEP quality assurance special study II l-A: 
2002 data replication - final design and management plan (FR-03-58). Alexandria, VA: Human 
Resources Research Organization. 

Wise, L.L., Sellman, S., & Sipes, S. (2003). Notes from the Meeting of the National Assessment 
Governing Board May 15-17, 2003 (SR-03-32). Alexandria, VA: Human Resources Research 
Organization. 

Hoffman, R.G., Becker, D.E., & Wise, L.L. (2003). NAEP quality assurance checks of the 2002 
reading assessment results for Delaware (FR-03-25). Alexandria, VA: Human Resources 
Research Organization. 

Hoffman, R.G., Wise, L.L., & Sticha, P. J. (2003). Review of NAEP quality control plans (TR-03- 
07). Alexandria, VA: Human Resources Research Organization. 

Hoffman, R. G. & Wise, L. L. (2003). The accuracy of school classifications for the 2002 
accountability cycle of the Kentucky Commonwealth Accountability Testing System (FR-03-06). 
Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., Becker, D.E., & Ramsberger, P.F. (2003). Report on past NAEP problems (FR-03- 
03). Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., Sipes, D.E., Harris, C.D., Ford, J.P., Sun, S., Dunn, J., & Goldberg, G.L. (2002). 
Independent evaluation of the California High School Exit Examination (CAMS EE): Year 3 
evaluation report (IR-02-28). Alexandra, VA: Human Resources Research Organization. 

Wise, L.L., Sipes, D.E., Harris, C.D., George, C.E., Ford, J.P., & Sun, S. (2002). Independent 
evaluation of the California High School Exit Examination (CAHSEE): Analysis of the 2001 
administration (FR-02-02). Alexandra, VA: Human Resources Research Organization. 

Wise, L.L., Sipes, D.E. (Sunny), George, C.E., Ford, J. P., & Harris, C.D. (2001). California High 
School Exit Examination (CAHSEE): Year 2 evaluation report (IR-01-29). Alexandria, VA: 

Human Resources Research Organization. 

Hoffman, R.G., & Wise, L.L. (2001 ). The accuracy of school classifications for the interim 
accountability cycle of the Kentucky Commonwealth Accountability and Testing System. (FR- 
01-26). Alexandria, VA: Human Resources Research Organization. 

McBride, J.R., Paddock, A.F., Wise, L.L., Strickland, W.J., & Waters, B.K. (2001). Testing via 
the internet: A literature review and analysis of issues for Department of Defense internet testing 
of the Armed Services Vocational Aptitude Battery (ASVAB) in the high schools (FR-01-12). 
Alexandria, VA: Human Resources Research Organization. 

Hoffman, R.G., & Wise, L.L. (2000). The accuracy of students’ novice, apprentice, proficient, and 
distinguished classifications of the Kentucky Core Content Test (FR-00-25). Alexandria, VA: 

Human Resources Research Organization. 

Hoffman, R.G., & Wise, L.L. (2000). School classification accuracy final analysis plan for the 
commonwealth accountability and testing system (FR-00-26). Alexandria, VA: Human Resources 
Research Organization. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 19 of 26 


Hoffman, R.G., Thacker, A., & Wise, L.L. (2000). The accuracy of students’ novice, apprentice, 
proficient, and distinguished ciassifications for the 2000 Kentucky Core Content Test (FR-00-41). 
Alexandria, VA: Human Resources Research Organization. 

Wise, L.L., Harris, C.D., Sipes, D.E., Hoffman, R.G., & Ford, J.P. (2000). High schooi exit 
examination (HSEE): Year 1 evaiuation report (IR-00-27r). Alexandria, VA: Human Resources 
Research Organization. 

Wise, L.L., Harris, C.D., Sipes, D.E., Collins, M.M., Hoffman, R.G., & Ford, J.P. (2000). High 
schooi exit examination (HSEE): Suppiementai year 1 evaiuation report (IR-00-37). Alexandria, 
VA: Human Resources Research Organization. 

Hoffman, R.G., & Wise, L.L. (1999). Estabiishing the reiiabiiity of student ievei ciassifications: 
Anaiytic pian and demonstration (FR-WATSD-99-34). Alexandria, VA: Human Resources 
Research Organization. 

Wise, L.L., Hoffman, R.G., & Thacker, A.A. (August 1999). Evaiuation of caiibration and 
equating procedures for the Fiorida State Assessment. (FR-WATSD-99-41). Alexandria, VA: 
Human Resources Research Organization. 

Wise, L.L., McCloy, R.A., & Quartetti, D.A. (1998). Indicators of student effort on the National 
Assessment of Educational Progress (DFR-EADD-98-58). Alexandria, VA: Human Resources 
Research Organization. 

McCloy, R.A., Russell, T.L., & Wise L.L. (Eds.). (1997). General Aptitude Test Battery (GATB) 
improvement project final report. Washington, DC: U.S. Department of Labor, Divisions of Skills 
Assessment and Analysis, Office of Policy and Research, Employment and Training 
Administration. 

Wise, L.L. (1997, June). Merging ASVAB and KIRIS on-demand scores: Report of preliminary 
results (LRS 97-4). Frankfort, KY: Kentucky Department of Education. 

Wise, L.L., Welsh, J., Grafton, F., Foley, P., Earles, J, Sawin, L., & Divgi, D.R. (1992). Sensitivi- 
ty and fairness of the Armed Services Vocational Aptitude Battery (ASVAB) technical 
composites. Monterey, CA: Defense Manpower Data Center. 

Wise, L.L., Chia, W.J., & Rudner, L.M. (1990). Identifying necessary job skills: A review of 
previous approaches. Washington, DC: Pelavin Associates, Inc. 

Wise, L.L., Peterson, N.G., Hoffman, R.G., Campbell, J.P., & Arabian, J.M. {^990). Army Synthetic 
Validity Project: Report of phase III results. Washington, DC: American Institutes for Research. 

Wise, L.L., McHenry, J.J., Chia, W.J., Szenas, P.L., & McBride, J.R. (1989). Refinement of the 
Computer Adaptive Screening Test (CAST). Alexandria, VA: U.S. Army Research Institute for 
the Behavioral and Social Sciences. 

Wise, L.L., Hough, L.M., Szenas, P.L., & Keyes, M.A. (1988). Phase I report: armed services 
applicant profile (ASAP) item fairness analysis. Washington, DC: American Institutes for Research. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 20 of 26 


Wise, L.L., McHenry, J.J., & Young, W.Y. (1986). Project A concurrent validation: treatment of 
missing data (RS-WP-86-08). Alexandria, VA: U.S. Army Research Institute for the Behavioral 
and Social Sciences. 

McLaughlin, D.H., RossmeissI, P.G., Wise, L.L., Brandt, D.A., & Wang, M. (1984). Validation of 
current and alternative ASVAB area composites based on training and SQT information on FY 
1982 enlisted accessions (Technical Report No. 651). Alexandria, VA: U.S. Army Research 
Institute for the Behavioral and Social Sciences. 

Card, J.J., & Wise, L.L. (1981). Teenage mothers and teenage fathers: The impact of early 
childbearing on the parents' personal and professional lives. In F.F. Furstenberg, R. Lincoln, & 

J. Menken (Eds.), Teenage sexuality, pregnancy, and childbeahng. Philadelphia: University of 
Pennsylvania Press. 

Wilson, S.R., Stancavage, F.B., & Wise, L.L. (1981). Synthesis of recent research on medical 
career decisions: A comparative study of two generations of physicians. Palo Alto, CA: 

American Institutes for Research. 

Steel, L.M., & Wise, L.L. (1 977). Designing a study of adult accomplishment and life quality. 

Palo Alto, CA: American Institutes for Research. 

Wise, L.L., McLaughlin, D.H., & Steel, L.J. (1977). The Project TALENT data bank handbook. 
Palo Alto, CA: American Institutes for Research. 

Wise, L.L., McLaughlin, D.H., & Gilmartin, K.G. (1977). The American citizen: Eleven years 
after high school. Volume II. Palo Alto, CA: American Institutes for Research. 

Gilmartin, K.G., McLaughlin, D.H., Wise, L.L., & Rossi, R.J. (1976). Development of scientific 
careers: The high school years. Palo Alto, CA: American Institutes for Research. 

Rossi, R.J., Bartlett, W.B., Campbell, E.A., Wise, L.L., & McLaughlin, D.H. (1975). Using the 
TALENT profiles in counseling: A supplement to the career data book. Palo Alto, CA: American 
Institutes for Research. 

Wilson, S.R., & Wise, L.L. (1975). The American citizen: Eleven years after high school. Palo 
Alto, CA: American Institutes for Research. 

Presentations 

Wise, L. L. (2015). The Standards for Educational and Psychological Tests: Implications for 
Peer Review of State Assessments. Workshop for state testing directors at the National 
Conference on Student Assessment, San Diego, CA. 

Wise, L. L. (2015). Psychometric Considerations for the Next Generation of Performance 
Assessment: What are the implications for state assessment programs? Paper presented at the 
National Conference on Student Assessment, San Diego, CA. 

Wise, L. L. (2015). Educational Measurement: What lies ahead? Presidential address to the 
annual meeting of the National Council on Measurement in Education. Chicago, IL. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 21 of 26 


Wise, L. L. (2015). Psychometric Considerations for the Next Generation of Performance 
Assessment: Modeiing, Dimensionaiity, and Weighting of Performance Task Scores. Paper 
presented at the annual meeting of the National Council on Measurement in Education, 

Chicago, IL. 

Wise, L. L. (2015). Test Design and Deveiopment Foiiowing the Standards for Educationai and 
Psychoiogicai Testing. Presentation to the annual meeting of the National Council on 
Measurement in Education. Chicago, IL. 

Wise, L.L. (2013). Different but Comparabie: Good enough for government work?. Paper 
presented at the annual meeting of the National Council on Measurement in Education. San 
Francisco, CA. 

Wise, L.L. (2012). Combining muitipie indicators of achievement and growth. Paper presented 
at the annual meeting of the National Council on Measurement in Education. Vancouver, BC. 

Wise, L.L. (2012). Prior Unking efforts: The best iaid pians .... Paper presented at the annual 
meeting of the National Council on Measurement in Education. Vancouver, BC. 

Wise, L.L. (2012). Revising our Standards for Educationai and Psychoiogicai Testing. Paper 
presented at the Association of Test Publisher’s Innovations in Test Design Conference. Palm 
Springs, CA. 

Wise L. (April 2011). Situating the Generaiizabiiity of Performance Assessments within a Vaiidity 
Framework. Paper presented at the 201 1 Annual Meeting of the National Council on 
Measurement in Education, New Orleans, LA. 

Wise, L. (April 2011). Aggregating Resuits from Through-Course Assessments. Paper 
presented at the 201 1 Annual Meeting of the National Council on Measurement in Education, 
New Orleans, LA. 

Wise, L. (April 2011). Aggregating Resuits from Through-Course Assessments. Paper 
presented at the National Conference on Student Assessment, Orlando, FL. 

Wise, L. (April 2011). The Evoiving U.S. Educationai System: Fiow Can i-O Psychoiogy 
Contribute? Paper presented at the annual meeting of the Society for Industrial and 
Organizational Psychology, Chicago, IL. 

Wise, L. (June 2011). Update on Revision of the Standards for Educationai and Psychoiogicai 
Testing. Paper presented at the National Conference on Student Assessment, Orlando, FL. 

Wise, L. (August 2011). Update on Revision of the Standards for Educationai and Psychoiogicai 
Testing. Paper presented at the annual convention of the American Psychological Association, 
Washington, DC. 

Wise, L.L. (2009) Revising our Test Standards: issues with increased Use of Tests for 
Accountabiiity. Presentation at the 2009 annual meeting of the American Psychological 
Association, Toronto, Canada. 

Wise, L.L. (2009, June) Revising our Test Standards. Presentation to the CCSSO National 
Conference on Student Assessment, Los Angeles, CA. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 22 of 26 


Wise, L.L. (2009, April) Revising our Test Standards: Issues for Work Place Testing. 
Presentation at the 2009 annual meeting of the American Educational Research Association, 
San Diego, CA. 

Wise, L.L. (2008). Validating Indicators of College Preparedness: Ready or Not? Presentation 
to the 2008 Reidy Interactive Lecture Series. Portsmouth, NH 

Wise, L.L. (September 2008). State Assessments Today: What State are We In? Presentation 
to the Conference on Educational Testing in America: State Assessment Achievement Gaps, 
Federal Policy and Innovations. Washington, DC. 

Wise, L.L. (June 2008). Strengthening K-1 2 Accountability Systems: What could be better than 
peer review?. Presentation to the CCSSO National Conference on Student Assessment, 
Orlando, FL. 

Wise, L.L., & Plake, B. (June 2008). Procedures for Revising the Test Standards. Presentation 
to the CCSSO National Conference on Student Assessment, Orlando, FL. 

Wise, L.L., & Rui, N. (2008, March). Computing and Communicating Test Accuracy for High- 
Stakes Decisions. Paper presentation at the 2008 annual meeting of the National Council on 
Measurement in Education, New York City, NY. 

Wise, L.L. (2007). Vertical Alignment. Paper presented at the CCSSO Large Scale Assessment 
Conference, Nashville, TN. 

Shen, X. (Stanford University; former HumRRO intern). Wise, L.L., and Becker, D.E. (2006, 
April). Analysis of School Effects. In L. Roberts (Chair), Evaluating the Impact of a High-School 
Graduation Test. Symposium conducted at the annual conference of the American Educational 
Research Association, San Francisco. 

Wang, X. , Wise, L.L., Becker, D.E., and Taylor, L.R. (2006, April). Comparison of CAHSEE 
Content Standards to Graduation and High-School Accountability Tests Used in Other States. In 
L. Roberts (Chair), Evaluating the Impact of a High-School Graduation Test. Symposium 
conducted at the annual conference of the American Educational Research Association, San 
Francisco. 

Wise, L. (Discussant) (2006, May). Expanding our influence: How I -O psychologists can 
improve education. Practice Forum conducted at the 21 Annual Conference of the Society for 
Industrial and Organizational Psychology, Dallas, TX. 

Wise, L.L. and Becker, D.E. (2006, April). Analysis of Results from Administrations of the 
California High School Exit Exam (CAHSEE). In L. Roberts (Chair), Evaluating the Impact of a 
High-School Graduation Test. Symposium conducted at the annual conference of the American 
Educational Research Association, San Francisco. 

Wise, L. W. (2004). Independent evaluation of the California High School Exit Exam (CAHSEE). 
Presentation at the American Educational Research Association Annual Meeting, San Diego, CA. 

Wise, L. W. (2004). Improving scientific research in education: Recent activities of the National 
Research Council. Presidentially Invited Symposium conducted at the American Educational 
Research Association Annual Meeting, San Diego, CA. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 23 of 26 


Wise, Lauress L. (2004). Vertically articulated content standards. Presentation for the Reidy 
Interactive Lecture Series. Nashua, NH. 

Wise, Lauress L. (2004). Meeting Alignment challenges: Analyzing vertical alignment. Paper 
presented at the CCSSO Large Scale Assessment Conference. Boston, MA. 

Wise, Lauress L. (2004). Independent Evaluation of the CAHSEE: Update on Evaluation 
Findings and Recommendations. Presentation to the California State Board of Education, 
Sacramento, CA. 

Wise, Lauress L. (2004). Debra P. v. Turlington and CAHSEE: Those who do not study history 
are doomed to repeat it: Presidential Invited Session. Annual Meeting of the American 
Educational Research Association. San Diego, CA. 

Wise, L. L., Floden, R.E., Dickersin, K. & Schneider, B.L. (2004). Improving Scientific Research 
in Education: Recent Activities of the National Research Council. Presidential Invited Session. 
Annual Meeting of the American Educational Research Association. San Diego, CA. 

Wise, L.L. (2001). Building a selection test battery: Validity, fairness, and other tradeoffs. 
Presentation to the Personnel Testing Council/Metropolitan Washington. Washington, DC: 
National Research Council. 

Wise, L.L. (2001). Validity, Fairness, and other Tradeofs. Presentation to the Personnel Testing 
Council/Metropolitan Washington. Washington, DC: National Research Council. 

Wise, L.L. (1 999). How far should NAEP go in serving an interpretive function? Presentation to 
the Forum on NAEP Design: 2000-2010. Washington, DC: National Research Council. 

Wise, L.L. & Hauser, R.M. (1999). Evaluation of the Voluntary National Tests. Paper presented 
at the annual meeting of the American Educational Research Association, Montreal, Canada. 

Waugh, G.W., Wise, L.L., Quartetti, D.A., & Ramos, R.A. (1999). Validation of the air traffic 
controller predictor tests. In R.A. Ramos, (Chair), Air traffic selection and training project. 
Symposium conducted at the 14th Annual Conference of the Society for Industrial and 
Organizational Psychology, Inc., Atlanta, GA. 

Wise, L.L., Quartetti, D.A. (HumRRO), Kieckhaefer, W.F. (RGI), & Houston, J.S. (PDRI). (1999). 
Development of air traffic controller predictor battery. In R.A. Ramos (Chair), Air traffic selection 
and training project. Symposium conducted at the 14‘^ Annual Conference of the Society for 
Industrial and Organizational Psychology, Inc., Atlanta, GA. 

Wise, L.L. (1999, April). How far should NAEP go in serving an interpretive function? 
Presentation to the Forum on NAEP Design: 2000-2010. Washington, DC: National Research 
Council. 

Wise, L.W. (1999). Test-taking motivation as persistence: Its effect on item and test 
performance. Paper presented at the Festshiftfor William Meredith, Berkeley, CA. 

Wise, L.L., Quartetti, D.A., Kieckhaefer, W.F., & Houston, J.S. (1999). Development of air traffic 
controller predictor battery. In R.A. Ramos (Chair), Air traffic selection and training project. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 24 of 26 


Symposium conducted at the 14th Annual Conference of the Society for Industrial and 
Organizational Psychology, Inc., Atlanta, GA. 

Wise, L.L. (1998, April). Relationship of Kentucky High School Assessment Scores to 
Predictors. In R.G. Hoffman (Chair), Symposium conducted at the Annual Meeting of the 
National Council on Measurement in Education, San Diego, CA. 

Wise, L.L. (1998, April). Discussant in Statistical support for CAT implementation. Paper 
Session at the Annual Meeting of the National Council on Measurement in Education, San 
Diego, CA. 

Wise, L.L. (1997). Career directions beyond the dissertation: R&D "think tanks." Presenter at 
invited symposium. Career directions in measurement: Beyond the dissertation, sponsored by 
NCME Graduate Student Issues Committee and co-sponsored by AERA Division D, at the 1997 
National Council on Measurement in Education Annual Meeting, Chicago, IL. 

Wise, L.L. (1996, April). A persistence model of motivation and test performance. Paper presented 
at the annual meeting of the American Educational Research Association, New York, NY. 

Curran L.T., & Wise, L.L. (1994). Enlistment processing changes. Paper presented at the 
annual meeting of the International Military Testing Association, The Netherlands. 

Welsh, J.R., & Wise, L.L. (1994). The stability of the Mantel-Haenszel odds ratio. Paper 
presented at the annual meeting of the International Military Testing Association, The 
Netherlands. 

Wise, L.L., & Welsh, J.R. (November 1993). Order adjustment and cross-correlation in ASVAB 
test form development. Paper presented at the annual meeting of the International Military 
Testing Association, Williamsburg, VA. 

Wise, L.L., & Wall, J.E. (1993, November). Plan for evaluating the ASVAB career exploration 
program. Paper presented at the annual meeting of the International Military Testing 
Association, Williamsburg, VA. 

Wise, L.L., & Curran, L.T. (1993, August). Introducing the new ASVAB: Recommendations and 
decisions for change. Paper presented at the annual meeting of the American Psychological 
Association, Toronto, Canada. 

Wise, L.L. (1993, April). Scoring rubrics for performance tests: Lessons learned from job 
performance assessment in the military. Paper presented at the annual meeting of the National 
Council on Measurement in Education, Atlanta, GA 

Wise, L.L. (1993, April). Test form accuracy. Paper presented at the annual meeting of the 
National Council on Measurement in Education, Atlanta, GA. 

Wise, L.L. (1992, April). Lessons learned from military performance assessment. Paper 
presented at the annual meeting of the National Council on Measurement in Education, San 
Francisco, CA. 

Wise, L.L. (1 991 , October). Overview of the ASVAB revision process. Paper presented at the 
annual meeting of the Military Testing Association, San Antonio, TX. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 25 of 26 


Wise, L.L., & McDaniel, M.A. (1991). Cognitive factors in the Armed Services Vocational 
Aptitude Battery and the General Aptitude Test Battery. Paper presented at the annual meeting 
of the American Psychological Association, San Francisco, CA. Wise, L.L., Chia, W.J., & Park, 
R.K. (1989). Effects of item position on IRT parameter estimates and item statistics. Paper 
presented at the annual meeting of the American Educational Research Association, San 
Francisco, CA. 

Arabian, J.M., Wise, L.L., & Szenas, P.L. (1988). Setting performance standards for Army 
enlisted jobs. Paper presented at the annual meeting of the American Psychological Associa- 
tion, New Orleans, LA. 

Szenas, P.L., Wise, L.L., & Arabian, J.M. (1988). Combining individual standards into an overall 
standard: Modeling the judgement process and investigating differences among judges. Paper 
presented at the annual meeting of the American Psychological Association, New Orleans, LA. 

Wise, L.L., McHenry, J.J., & Campbell, J.P. (1987). Matching skills and traits to job require- 
ments: Results from Project A. Paper presented at the annual meeting of the American 
Educational Research Association, Washington, DC. 

Wise, L.L. (1987). Differential item difficulty indicators in small samples. Paper presented at the 
annual meeting of the American Educational Research Association, Washington, DC. 

Wise, L.L., McHenry, J.J., RossmeissI, P.G., & Oppler, S.H. (1986). ASVAB validities using 
improved job performance measures. Paper presented at the annual meeting of the Military 
Testing Association, Mystic, CT. 

Wise, L.L., Campbell, J.P., McHenry, J.J., & Hanser, L.M. (1986). A latent structure model of job 
performance factors. Paper presented at the annual meeting of the American Psychological 
Association, Washington, DC. 

Wise, L.L. (1986). Latent trait models for partially speeded tests. Paper presented at the annual 
meeting of the American Educational Research Association, San Francisco, CA. 

Wise, L.L., & Mitchell, K.J. (1985). Development of an index of maximum validity increment for 
new predictor measures. Paper presented at the annual meeting of the American Psychological 
Association, Los Angeles, CA. 

Wise, L.L., & Wilson, S.R. (1982). Test item calibration. Paper presented at the annual meeting 
of the American Educational Research Association, New York, NY. 

Wise, L.L., & McLaughlin, D.H. (1981). Survey data enhancement. Paper presented at the 
annual meeting of the American Educational Research Association, Los Angeles, CA. 

Wise, L.L., Wilson, S.R., & Stancavage, F.B. (1980). The development of medical practice and 
residence value scales that distinguish physicians in different specialties and practice locations. 
Paper presented at the annual meeting of the American Educational Research Association, 
Boston, MA. 

Wise, L.L., & Steel, L.M. (1979). The effects of school quality on student's knowledge and skills. 
Paper presented at the annual meeting of the American Educational Research Association, San 
Francisco, CA. 



Case l:14-cv-00857-TSC Document 60-73 Filed 12/21/15 Page 26 of 26 


Wise, L.L. (1979). Long-term consequences of sex differences in high school mathematics 
education. Paper presented at the annual meeting of the American Educational Research 
Association, San Francisco, CA. 

Steel, L.M., & Wise, L.L. (1979). Origins of sex differences in high school mathematics 
achievement and participation. Paper presented at the annual meeting of the American 
Educational Research Association, San Francisco, CA. 

Wise, L.L. (1978). The role of mathematics in women's career development. Paper presented at 
the annual meeting of the American Psychological Association, Toronto, Canada. 



